A New Method for Rule Finding Via Bootstrapped Confidence Intervals

Association rule discovery in large data sets is vulnerable to producing excessive false positives, due to the multiple inference effect. This paper first sets this issue in precise mathematical terms and presents some analytical results. These show that a common concern regarding effects of filtering is not as problematic as had been previously thought. The analytical results also shed new light on a recently proposed method for dealing with the problem. The paper then proposes a new method based on simultaneous confidence intervals, computed via a novel use of the statistical bootstrap tool. The proposal here differs markedly from previous bootstrap/resampling approaches, not only in function but also in basic goal, which is to enable much more active participation by domain experts.